Purely competitive evolutionary dynamics for games 
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We introduce and analyze a purely competitive dynamics for the evolution of an infinite population 
subject to a 3-strategy game. We argue that this dynamics represents a characterization of how 
certain systems, both natural and artificial, are governed. In each period, the population is randomly 
sorted into pairs, which engage in a once-off play of the game; the probability that a member 
propagates its type to its offspring is proportional only to its payoff within the pair. We show that 
if a type is dominant (obtains higher payoffs in games with both other types), its 'pure' population 
state, comprising only members of that type, is globally attracting. If there is no dominant type, 
there is an unstable 'mixed' fixed point; the population state eventually oscillates between the 
three near-pure states. We then allow for mutations, where offspring have a non-zero probability of 
randomly changing their type. In this case, the existence of a dominant type renders a point near 
its pure state globally attracting. If no dominant type exists, a supercritical Hopf bifurcation occurs 
at the unique mixed fixed point, and above a critical (typically low) mutation rate, this fixed point 
becomes globally attracting: the implication is that even very low mutation rates can stabilize a 
system that would, in the absence of mutations, be unstable. 

PACS numbers: 02.50.Le, 05.45.-a, 89.65.-s, 87.23. Gc, 87.23.Kg 



I. INTRODUCTION 

A relatively recent development in the game theory 
literature, that of the dynamic treatment of games, has 
proven successful in providing answers both to the ques- 
tion of which equilibria will arise in a given game, and 
how they might be reached; both of these questions have 
proven problematic in the static, introspective approach 
to game theory The fundamental idea behind the 
dynamic treatment is to consider repetitions of a game, 
rather than a once-ofF play, thus allowing players' strate- 
gies to evolve through time. Rules are posited to govern 
the evolution of the players' strategies, with such rules 
being either mindless (occurring exogenous to any deci- 
sion making of the players - we call such a rule a 'dy- 
namics') or minded (where the rule depends explicitly 
on players' decisions - we call such a rule a 'learning 
rule'). Different interpretations of the same game might 
render the same rule either mindless or minded, so that 
the distinction is not clearly set. 

Our focus in this paper will be on a particular rule, 
which, as we shall discuss, can be interpreted cither as a 
dynamics or a learning rule. Such rules are often most 
comfortably cast in the language of (biological) evolu- 
tion, and it is such a description, for the most part, that 
we shall employ. Typically, a population of agents is con- 
sidered, with each agent being one of a number of types 
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(the types correspond to strategies). In each period, a 
subset of the population is paired up, with each pair en- 
gaging in a once-off play of the game (which might be 
termed a fight, a mating, etc., depending on the general 
interpretation). The result of the pairings and engage- 
ments determines the make-up of the population in the 
next period, when the process is repeated. 

The dynamics thus provides a vehicle by which an out- 
come of the game (the long term make-up of the popu- 
lation, for example) may be reached. For many sensibly 
founded dynamics (there are conditions for what consti- 
tutes a 'sensible' dynamics, typically based on the general 
principle that relative success of a certain type/strategy 
should lead to its increase as a proportion of the popula- 
tion - for more technical definitions, a good reference is 
Samuclson Q) this outcome tends to correspond to some 
of the equilibrium concepts defined for once-off games, al- 
though it is not unusual for this not to be the case. That 
is, a sensible dynamics may result in an equilibrium that 
is not even Nash (see later in the present paper, as well 
as Fudenberg and Levine [l|). 

It is instructive, in terms of understanding the nature 
of a typical dynamics and also to provide a foundation 
and a foil for the model that we develop in the main part 
of the present paper, briefly to describe what is proba- 
bly the most well-known and widely-applied dynamics, 
the 'replicator dynamics', first introduced by Taylor and 
Jonker An infinite population comprising n types of 
agent is assumed; the population state is defined as the 
proportions of each type in the population: 

P= (Pl,P2,---,Pn) G A", 
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where A" is the unit simplex in R": 

A" |p G M" : p > A = l| . 

A type's 'fitness' at a given time is defined as the ex- 
pected payoff to one of its agents in a once-off game with 
a randomly selected opponent from the population; the 
proportional growth rate of a given type's proportion at 
a given time is equal to the difference between that type's 
fitness and the average fitness throughout the entire pop- 
ulation. 

The key concept behind the replicator dynamics is of 
the fitness of types within the population. This is deter- 
mined not only by how a certain type fares against other 
types, but also by how well agents of that type do in en- 
gagements with agents of the same type. Indeed, this is 
a vital consideration in the concept of evolutionary sta- 
bility under the replicator dynamics - a population state 
comprising only one type of agent is resistant to inva- 
sions by other types if, and only if, the predominant type 
fares better against itself than the potential invader does 
against the predominant type Q . This is clearly a sensi- 
ble consideration in modeling certain systems; it has been 
argued that the replicator dynamics provides a model for 
some aspects of evolution in the natural world [sj , where 
a type whose agents do very poorly amongst themselves 
would not be expected to succeed in increasing as a pro- 
portion of the population, while a type whose agents do 
very well amongst themselves would be expected at least 
to be able to maintain a high proportional representation 
in the population. 

However, in certain circumstances, this 'cooperation' 
consideration is not a natural one; there exist instances 
where the success of a type does not depend at all on 
how well agents of that type do amongst each other, but 
only on how well agents of that type do against agents of 
other types. It is dynamics of this type that we consider 
in the present paper. 

'Purely competitive dynamics', as we shall term them, 
arise most directly in a knock-out tournament where a 
large population comprising a certain number of types 
is paired up in each round, with each pair engaging in a 
once-off game from which only one of each pair advances, 
with a contestant's probability of advancing depending 
on his relative payoff in his engagement. Here, the suc- 
cess of a given type (as measured by the proportion of 
remaining contestants that are of that type) is invari- 
ant to how agents of that type fare against each other. 
Purely competitive dynamics could also model a learning 
rule where, in a population of distinct behavioral types, 
offspring emulate the behavior of their more successful 
parent. If an offspring's parents are of the same type, 
it is clear that the offspring will be of that type; there 
is propagative competition only between parents of dif- 
fering types. Later in the paper, we shall discuss these 
examples in the specific context of our model. 

A relatively new consideration in the game theory lit- 
erature is that of the effect of mutations (generally speak- 



ing, the random changing of types within the population, 
though mutations have been variously defined) on the dy- 
namics of evolutionary models. The interest in the effect 
of mutations lies not just in their natural applications - 
mutation at a genetic level is one of the key features driv- 
ing biological evolution Q - but also in their interpreta- 
tion in terms of imperfect memory, experimentation, and 
bounded rationality 0. Mutations have been shown to 
have qualitatively significant effects on the long-run dy- 
namics of a number of models [see e.g. Refs. I8l-[l0l|: in 
this paper, we shall give some attention to the effect of 
mutations on our competitive dynamics. 

The balance of the paper is structured as follows. Sec. 
im sets out the general model - a sensible characteriza- 
tion of purely competitive dynamics as described above 
- while the sections thereafter investigate the behavior 
of the population state under the dynamics in the con- 
text of 3-type games (governed by 3 x 3 matrices) , which 
have the simultaneous benefits of offering a rich set of 
possible dynamics and of being easily characterized and 
illustrated. Sec. [Hi] analyzes the dynamics without muta- 
tions; we show that, under some simplifying assumptions, 
the behavior of the population state can, qualitatively, be 
fully characterized. In Sec. IIVI we investigate the effect 
of mutations on our dynamics. We show that even low 
mutation rates can have a nontrivial effect on the behav- 
ior of the system; the nature of this effect is typically to 
stabilize the system in a way that, again, is qualitatively 
understood. Finally, Sec. |V] concludes, and offers some 
possibilities for future study. 



II. THE MODEL 
A. Description of model 

We assume a population of N agents, each of which 
is assigned a pure strategy from the strategy set 
S = {si,S2,S3}; an agent will be said to be 'of type i' 
if it is assigned s,; . The state of the population in period 
t is defined by the proportions of the respective types 
in the population; we shall denote the population state 
at time t by pt := {pit,P2t,P3t) & A^. The 3x3 payoff 
matrix for the game to which the strategies are relevant 
is given by U, so that the payoff to a type i agent in a 
once-off play with a type j agent is ity . We require that 
the elements of U be strictly positive, and that no two 
opposite elements be equal (the first of these assumptions 
is fundamental, as will shortly be seen; the second is not 
as important, but is rather for convenience). 

In each period, a proportion of the population is se- 
lected and randomly arranged into pairs. The proportion 
of the population that is selected in a given period will 
depend on the individual probabilities of selection of the 
three types, which in turn depend on the population state 
in the period. Defining : A^ — )■ [0,1], we denote by 
(f>i{pt) the probability that a type i agent is selected for 
pairing in period t. Two possibilities immediately present 
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themselves for consideration. The first is (pi = 1 \/ i, i.e. 
that in each period, regardless of the population state, 
each agent is selected for pairing. In a biological context, 
this would be referred to as a population in which there 
is zero 'selection pressure' The second is that an 

agent's probability of selection is proportional to its ex- 
pected payoff - the strength or fitness of its type in the 
given population state ~ so that (j)i{p) oc e^Up (e^ G M'' 
denotes the i*'^ canonical basis element of M'^). 

Once the pairs are selected, each pair engages in a 
once-off play of the game, the outcome of which is the 
production of two offspring, who replace their parents in 
the population and enter into the next period (so that the 
population size remains unchanged). The two offspring 
of a given pairing arc of the same type, with that type 
determined as follows: 

prob [offsp. type k \ parents type = 

where > is a constant, and 5ab is the Kronecker delta 
i^ab = 1 if a = 6, else 6ab = 0). It should be noted 
here that this expression is not arbitrary; it is motivated 
primarily by the intuitive appeal of two cases, 9 = 1 
and 9 = oo. Taking 9 = 1, as we shall throughout this 
paper, the probability that the offspring of a pairing of 
a type 1 agent and a type 2 agent are of type 1, for 
example, is proportional to the payoff the type 1 parent 
receives in the pairing. On the other hand, in the limiting 
case 9 = oo, the offspring assume with certainty the type 
of the parent with the higher payoff. The parameter 9 
therefore governs the transmission from relative strength 
in a mating pair to probability of propagating one's type 
in mating. 

Often, studies of evolutionary dynamics allow for mu- 
tation. We generalize our model by assuming that each 
offspring, after it inherits a type from its parents but be- 
fore it enters the population for the next period, changes 
its type with (fixed) probability fi > 0, with each of the 
other two types equally likely. 

In the analysis that follows, for convenience, we shall 
assume an infinite population, i.e. N = oo; the finite- 
population case is less tractable, and we relegate its con- 
sideration to a brief discussion in the concluding section. 
We shall also focus exclusively on the case (j>i = 1; the 
case of (j)i{p) oc eiUp will be investigated in a later study 
by the authors. Finally, as already mentioned, we assume 
9=1 throughout. 



B. Possible interpretations 

Though not specifically constituted to emulate a bi- 
ological process, the model developed above does lend 
itself to a number of biological interpretations. 

Assume, for example, that each agent represents an an- 
imal that is genetically programmed to play some pure 
strategy, i.e. to exhibit a specific behavior or trait. The 



model could then describe the dynamics of intrasexual se- 
lection within a population [see e.g. Refs.[T2|-[l^: a single 
play of the game pits two animals - and indirectly, the 
genes that code for their respective behaviors - against 
each other; the animals whose strategies receive higher 
payoffs within their pairings typically go on to mate and 
to propagate their genetic material to the next genera- 
tion, while the weaker animals typically do not. For con- 
sistency with the assumption of a fixed population size, 
we could assume full generational replacement and that 
victorious animals each produce two offspring, as sug- 
gested in Sec. Ill Al alternatively, we could assume that 
competition eliminates the losing animals from the pop- 
ulation, in which case each victor need only produce a 
single offspring. (To obviate such contrivances, we might 
simply generalize the model to allow for a variable pop- 
ulation size.) The parameter 9 governs the probability 
that a weaker competitor beats a stronger competitor: 
for 9 = oo, stronger competitors will always emerge vic- 
torious, whereas for < 6' < oo, there is a nonzero prob- 
ability of weaker competitors beating stronger ones (on 
average, though, stronger competitors will still win more 
often). A number of interpretations for jj, > are possi- 
ble, perhaps the simplest being that a nonzero mutation 
rate corresponds directly to random mutation at a ge- 
nomic level. 

In any event, the competitive nature of the dynamics is 
clear: the success of a given strategy depends only on how 
well it fairs against other types. This is a sensible charac- 
terization of intrasexual selection: if two animals playing 
the same strategy are competing for a mate, the payoff 
within the interaction is irrelevant in the sense that, one 
way or another, the set of genes that codes for their com- 
mon strategy (behavior, trait, etc.) will be propagated 
to the next generation. Of course, if the animals survive 
for more than one generation, and generations are suffi- 
ciently close together, a low common payoff in encoun- 
ters with their same type might render them weaker in 
subsequent generations; if this is the case, the replicator 
dynamics (for example) might be more appropriate. 

Though the natural description of our model makes 
use of biological terminology ('offspring', 'parents', 'mat- 
ing', etc.), there are many more general competitive in- 
teractions which could conceivably be described by the 
same model. Consider a boxing tournament, for exam- 
ple. Agents playing the three different strategies could 
correspond, conveniently, to the three commonly defined 
styles of boxer: the 'swarmer', the 'slugger', and the 'out- 
fighter' Boxers arc paired up at each stage, with 
one of the boxers in each pairing advancing to the next 
round, and the other being eliminated; our dynamics fea- 
tures the sensible condition that a boxer's probability of 
advancing to the next round is dependent purely on that 
boxer's strength relative to his opponent. (To keep the 
number of boxers constant at TV, we could specify that, 
in advancing to the next round, a boxer earns two fights 
for that round.) 

Furthermore, as already noted, the distinction between 
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mindless and minded rules is not clearly set, and our 
model can be thought of as describing a process of social 
or observational learning [l6l - [l8| . rather than a mind- 
less competitive interaction (somewhat surprisingly, the 
modeling of such social behavior is of increasing concern 
in the context of statistical physics [l^ ) . 

In the social learning paradigm, players can learn only 
from existing players in the population, i.e. by observ- 
ing the interaction between other players, or by 'asking 
around'. For example, we might suppose that each gener- 
ation, new agents (perhaps, but not necessarily, offspring 
of current agents) enter the population, with each new 
agent making a once- and- for- all choice of strategy based 
on an observation of a play of the game between an exit- 
ing agent and one other agent drawn randomly from the 
population. Sensibly, the new agent will be more likely 
to emulate the strategy which received the higher payoff 
in the observed interaction: for 9 = oo, the new agent 
will emulate with certainty the strategy observed to be 
more successful; 9 < oo could be explained in terms of 
imperfect observation, while > could be explained in 
terms of experimentation. 

Alternatively, instead of assuming that agents are peri- 
odically replaced, we could assume that agents are mem- 
oryless or that they consider past experience irrelevant 
to their current situation: for example, an agent might 
stick to its strategy until it is paired up with an opponent 
who receives a higher payoff than it does, in which case 
it will adopt the strategy of its opponent [l[ . 

Given these additional interpretations in an observa- 
tional learning framework, the dynamics might well be 
referred to as an 'emulation dynamics'; nevertheless, the 
competitive nature of the dynamics remains clear. 



III. THE CASE OF NO MUTATIONS 

As noted, we make the following assumptions through- 
out the paper: 4>i{p) = 1 V p e A^,i = 1,2,3; 6* = 1; 
and N ^ oo. In this section we shall also assume fj, = 
(no mutations). Some noteworthy features of the game 
under these assumptions arc listed below. 

1. Since c/)i(p) = 1 for i = 1,2,3, the diagonal ele- 
ments of the payoff matrix U are irrelevant: when 
a type i parent meets another type i parent, their 
offspring are guaranteed to be of type i. Thus, we 
are free to normalize the diagonal elements of U as 
we wish - the dynamics will be unaffected. 

2. Multiplying opposite elements of U by the same 
factor leaves the effects of the dynamics unchanged; 
the probability that an offspring is of type i, if its 
parents are of types i and j, is invariant to multi- 
plying Uij and Uji by some factor A. This factor can 
be unique for different pairs of opposite elements. 
Thus, we are free to scale opposite elements in the 
payoff matrix without fear of altering the dynamics. 



It will turn out that a convenient normalization of the 
payoff matrix U — >■ U is given by: 



LL2J • 

This normalization obeys the conditions set out in notes 
(1) and (2) above ~ it is equivalent to setting all diagonal 
elements to ^ , and scaling off-diagonal opposite elements 
Uij and Uji by the factor A = {uij + Uji)~^ - so that the 
trajectories of the population state under U will be iden- 
tical to those under U. The benefit of this normalization 
is that, for i j, Uij is simply the probability that a 
pairing of a type i agent and a type j agent will yield 
offspring of type i. 

A. Stage-to-stage dynamics 

Assume that the population state at time t is pt = 
{pu,P2t,Pzt)- Consider the possible pairings that could 
give rise to type 1 offspring, assuming that there are no 
mutations (/^ = 0): 

• type 1 - type 1, giving rise to type 1 offspring with 
probability 1 - the probability that a random pair- 
ing is of this type is p^^; 

• type 1 - type 2, giving rise to type 1 offspring with 
probability ui2 - the probability that a random 
pairing is of this type is 2pitP2t] and 

• type 1 - type 3, giving rise to type 1 offspring with 
probability ui^ - the probability that a random 
pairing is of this type is 2pitP3t- 

In an infinite population, the proportion of each pair- 
ing possibility amongst all other possibilities corresponds 
exactly with its probability; also, for a given pairing type- 
combination, the proportion of such meetings that result 
in type 1 offspring corresponds exactly to the probabil- 
ity that a pairing of that type-combination yields type 
1 offspring. Thus, in the case of an infinite population 
with no mutations, we have the following exact update 
formula for pi : 

Pl,t+1 = pit + 2pitP2tWl2 + 2pitP3tUl3. 

(For the finite population case, we would have to make 
use of expected value operators.) 

The update formulae for p2 and p^ are reached 
similarly, and we arrive at the update formula for the 
population state as a whole: 

{Pl.t+l = pit + 2pitP2tUi2 + 2pitP3tUi3 
P2.t+1 = pit + 2p2tPltM21 + 2p2tP3tU23 (1) 
P3,t+1 = pit + 2p3tPltU31 + 2p3tP2tU32- 
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In Sgc. llVl we shall augment the above update formula 
to account for mutatfons; in that case, offspring of a given 
type can result from any pairing. 

B. Fixed points and their stability 

A fixed point is a population state which, under the 
dynamics, persists through time. 

Definition 1. The population state p is a fixed point if 

Pt = P Pt+i = P- 

Note that the assumption of an infinite population al- 
lows us to ignore the stochastic element of the dynamics 
in characterizing such points. 

In discussing the stability of fixed points, a key con- 
cept is that of 'evolutionary stability' [20,[21[. Loosely, 
a fixed point exhibits evolutionary stability if, after the 
introduction into the population of any group of agents 
of slightly different proportional make-up, the population 
state moves back to the initial population state (the fixed 
point). In other words, for all perturbations smaller than 
a given size, the population state eventually returns to 
the fixed point. 

Definition 2. Let p be a fixed point, p is evolutionarily 
stable (hereafter 'ES') if 3 e > such that 

PteB^3{p,e) => lim pt+^ = p. 

r— )-oo 

(Here, as hereafter, we denote by Sa" (p, e) the open 
ball on A" of radius e, centered at p € A"; that is, 
BA"(p,e) := {x e A" : ||x — p|| < e}, where || • || is the 
Euclidean norm in M".) 

Finally, we shall call a fixed point 'globally attracting' 
if, for any initial population state away from the other 
fixed points, the population state converges to that fixed 
point. 

Definition 3. Let p be a fixed point, along with 
p-*-, p^, . . . , p™. p is globally attracting if: 

Pt e A3\{pi,...,p™} ^ limpt+, = p. 

r — >oo 

It is clear that any globally attracting isolated fixed 
point is also ES. 

We shall begin our identification and classification of 
the fixed points with the following trivial result: 

Theorem 1. // /i ~ 0, the population states ei,e2 and 
e3 are fixed points. 

Proof. If there are only type i agents in the population 
(p = Bi), pairings comprise only type i agents, and thus, 
with no mutations, only type i offspring will be produced. 
This state will persist. □ 



We shall now consider the evolutionary stability of 
these 'pure' states. We shall describe the conditions un- 
der which a pure state can be ES, and show that ES 
pure states are in fact globally attracting as well. We 
shall then show that at most one of the pure states can 
be ES, and describe the conditions under which none of 
the pure states is ES. 

Theorem 2. The fixed point p = e^ is ES if, and only 
if, Uij > \ i ^ i (equivalently, Uij > Uji V j ^ i). 
Moreover, if Uij > ^ V j 7^ i, then 61,62, are the only 
fixed points, and p = e^ is globally attracting. 

Proof. (<;=) Without loss of generality, assume U12, ^13 > 
^, and let the current population state be pt = 
{pit,P2t,P3t) 7^ 62, ea. We shall demonstrate that 
lim pi^t+T = 1; that is, the population returns to the 
pure state ei. 

From ID), we have the following update formula for pi: 

Pi,t+i = pit + 2pitP2tWl2 + '^PltPStUu 
= Pit [pit + 2p2tUl2 + 2p3tUi3) 
>Plt{pit + 2p2t{^)+2p3ti^)) 
= Pit {pit +P2t +P3t) ^Plt- 

We have equality in the third line if, and only if, 
P2t = Pat = (we have already assumed pu 7^ 0). 
So for all non-pure population states at time t, the 
Pi component is strictly monotonically increasing over 
time, so none is a fixed point, and thus lim pi.t+r ~ 1- 

The population state ei is therefore globally attracting, 
and by implication, ES too. 

(=>) Suppose, without loss of generality, that U12 < \. 
We shall show that the pure state p = ei is not resistant 
to the invasion of type 2 agents; in particular, we shall 
show that, for any population state pt = ^e2 + (1 — 
where ^ g (0, 1), the component pi decreases in time. 

The update formula for the pi component of p in a 
population where pit,P2t > and p^t = is given by: 

Pi,t+i = Pu + 2pitP2tMl2 

= Plt [pit + 2p2tUl2) 

< Pit {pit + 2p2t{l)) 

= Pit {pit +P2t) = Pit] 

that is, pi is strictly monotonically decreasing in time. 
So, any perturbation away from the initial state p = ei 
along that support of A'^ where ps ~ will ensure that 
the population state does not return to ei. Since any 
open ball Bj\3{ei,£) contains such points, the pure state 
p = ei is not ES. □ 

Fig. [IJa) on pg. [7] contains a phase portrait showing 
some typical trajectories for the case of an ES pure state. 
In this 'ternary diagram', as hereafter, the right, top and 
left vertices of the simplex correspond to the pure pop- 
ulation states ei,e2 and 63 respectively. The scaling for 
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each component of the population state is along the per- 
pendicular dropped from the vertex corresponding to its 
pure state to the side opposite that vertex. 

The case Wi2,ui3 > 5, which the previous theorem 
shows to be both necessary and sufficient for the pure 
state p = ei to be ES, does not permit any other pure 
state to be ES. (For 62 to be ES, the above theorem 
would require U21 > ^, which contradicts U12 > ^. The 
requirements for 63 to be ES lead to a similar contradic- 
tion.) So we can have at most one pure state ES fixed 
point. We shall call the type associated with such a state 
the 'dominant type'; the conditions required of the payoff 
matrix for its existence and the result in Thm. [2] justify 
this nomenclature. 

It is interesting to note that the evolutionarily stable 
fixed point in this case need not correspond to a Nash 
equilibrium of the static game; indeed, the globally at- 
tracting pure state can even correspond to a strategy 
that is strictly dominated in the static game! Take, for 
example, the following payoff matrix: 



U 



Under our dynamics, the pure state of type 1 agents is 
globally attracting, despite the fact that strategy 1 is 
strictly dominated by strategy 2. This may be under- 
stood by noting that the competitive dynamics requires 
us to consider opposite elements in the payoff matrix 
rather than those above and below each other, as the 
traditional concept of dominance does. 

Another consequence of Thm.[5]is that there exist cases 
where no ES pure state exists. These are the cases where 
none of the types is dominant: 




(i) U12 > i,?il3 < ^ "~ 



2 ' "23 > 5; 



(ii) U12 < 5,Ul3 > ^,"23 < 5- 



21 "13 2' "■^'^ ^ 2' 

A suitable permutation of the indices equates the two 
cases, so we can take, say, the first to be general. Here, 
type 1 'beats' type 2, which 'beats' type 3, which 'beats' 
type 1. In this case, the game mirrors the classic rock- 
papcr-scissors game of the game theory literature - see, 
for example, Hofbauer and Sigmund 1J|. 

We now turn our attention to non-pure state fixed 
points. We know from Thm. [2] that such fixed points 
are not possible if one of the fixed points is ES, so we 
need only consider the case where no ES pure state ex- 
ists, the general form of which we have taken to be case 
(i) above. 

The theorem below establishes that, in such a case, a 
unique non-pure state fixed point always exists, and that 
it lies in the interior of A'^ . The subsequent theorem will 
show that this interior fixed point can not be ES. 

Theorem 3. If uri > 5, ■1*13 < 5,1x23 > 5, then: 

(a) 31 p ^ A^\{ei,e2,e^} such that pt = p => Pt+i = 

p; 



(b) p e int(A3). 

Proof. We shall derive the conditions for such a fixed 
point to exist, and demonstrate that a unique point in 
A'^\{ei, e2, ea} fulfils these conditions. Assume that p e 
int(A'^) is our candidate for an interior fixed point. (We 
shall consider the case where p S bd(A'^)\{ei, e2, 63} 
later, and demonstrate that such a point can not be 
fixed.) Applying the update formula for pt-i-i, and sub- 
stituting in the fixed point condition that pt+i = Pt = P, 
we arrive at the following system of equations: 

2piP2Ml2 + 2piP3Mi3 
2p2PlU21 + 2p2P3M23 
2p3PlW31 + 2p3P2M32 




2P2M12 
2piM21 
2piM31 



2p3Ul3 
2P3M23 
2P2M32 



2Up= (1,1,1) 



where we used the fact that the components of p are 
nonzero in dividing them out in the second step. Taking 
into account the conditions pi-|-p2+P3 = 1 and Uij+Uji ~ 
1, we arrive at the following linear system: 

1/2 U12 ui3 

1 - U12 1/2 lt23 
1 - Ul3 1 - -1*23 1/2 
1 11 



The above system turns out to be consistent; the unique 
solution is given by: 




Pi 



P2 = 



P3 



1 - 2U23 



1 


- 2(U12 


- -"13 - 


\- U23) 




2u 


13-1 




1 


- 2(mi2 


- U13 ' 


\- U23) 




1 - 


- 2ui2 




1 


- 2(U12 


- ""13 ' 


\- U23) 



(2) 



It remains to be shown that pi,p2,P3 > 0. Note that 
U12 - ui3 + U23 > 5, so that 1 - 2('iii2 - ^13 + "23) < 0. 
We thus require the numerators above to be negative; it 
is readily verified that this is the case. Thus, p € int(A'^), 
and so, under the assumptions concerning U that forbid 
an ES pure state, there always exists a unique fixed point 
in the interior of A'^ ; it is given by ([2]) . 

Finally, we consider the case where p G 
bd(A'^)\{ei, 62, 63}. Assume, without loss of gen- 
erality, that P3 = 0, so that pi,P2 > 0, with pi+ P2 = 1- 
Our system of equations incorporating the fixed point 
condition is then 



Pi 



P\ 

P2 =P\ 

1 =Pl 4 



V 2piP2Ui2 
I- 2p2PlU2l 
P2 



(1 
(1 



2ui2)pi 
2ui2)Pi 



2ui2 



This is consistent only if iii2 — \; which we have assumed 
not to be the case. Thus, the only fixed points on the 
boundary of A'^ are the pure states. □ 
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(a) /i = 0.0; Si2 = 0.60, ui:i = 0.55, U23 = 0.40 (b) fj. = 0.0; U12 = 0.58, uia = 0.44, U23 = 0.62 




FIG. 1. A few representative trajectories for the case (a) where the existence of a dominant type leads to an ES pure state; 
and a typical trajectory for the case (b) where no dominant type exists, resulting in a locally-unstable interior fixed point. 



So wc have found the conditions under which an in- 
terior fixed point exists; it remains to characterize the 
stability of this interior fixed point. The following result 
establishes that the interior fixed point is never ES. 

Theorem 4. // an interior fixed point p exists, then 
V e > 0, 3 p' e B^3 (p, e) such that Pt = p' => 
lim pt+r ^ p. 

r— >-oo 

Proof. Suppose, without loss of generality, that U12 > 
^,ui3 < ^,""23 > ^, so that we have the interior fixed 
point given by ([2]), which we call p € int(A^). 

Taking into account the restrictions that the propor- 
tions sum to unity, and that opposite elements of U sum 
to unity, the third equation in ([1]) becomes redundant, 
and the system may be written in two dimensions: 

Pi,t+i = Pit bit(l - 2iti3) + 2p2t(Mi2 - U13) + 2U13] 
= : Fi{pit,P2t); 

P2,t+1 = P2t [2pit(l - U12 - U23) +P2t(l - 2U23) + 2it23] 

= : F2{pit,P2t)- 

p, understood in this context to be the vector compris- 
ing the first two elements of the original three-element 
p, is a fixed point of this system. To determine the 
stability of this fixed point, we employ the usual lin- 
earization method [e.g. Ref. [l^: if we can show that 
at least one of the eigenvalues of the Jacobian matrix 
d{Fi,F2)/d{pit,P2t), evaluated at the fixed point, has 
modulus greater than unity, then the fixed point is un- 
stable. Defining D := 1 — 2({ti2 — U13 + U23) < 0, these 
eigenvalues may, with some effort, be shown to be: 

, . / (2ai2-l)(l-2^i3)(2^23^ ,,./- 

Ai,2 = 1 ± « Y — = 1 ± zVw, 



where < w := -(2iii2 - 1)(1 - 2?ii3)(2u23 - 1)/D < 
1. So, the eigenvalues are distinct and complex, with 
modulus greater than unity: 

|Al,2p = l+CJ > 1. 

The fixed point is therefore locally unstable, i.e. not ES, 
and the trajectories nearby are outward spirals. On the 
unit simplex A^, these trajectories translate into outward 
spirals. □ 



A typical outward-spiraling trajectory is presented in 
Fig.Hb). 

A numerical investigation such as that carried out in 
the following section (and explained in greater detail 
there) reveals this outward spiral to have as a limit cycle 
the boundary of the simplex. The limiting behavior of 
the system, then, is oscillatory: the proportion of type 
1 agents increases to nearly unity (with a near zero pro- 
portion of type 2 and type 3 agents) , after which the pro- 
portion of type 2 agents increases to nearly unity, after 
which the proportion of type 3 agents increases to nearly 
unity, and so on. This oscillatory long-run behavior of 
the population state is similar to that under the replica- 
tor dynamics for rock-paper-scissors games Q ; this is un- 
surprising because, as previously noted, the payoff logic 
behind the current case in our dynamics mirrors that of 
the rock-paper-scissors game. 

Under some observational learning interpretations of 
our model, this oscillatory behavior might be regarded 
as unrealistic: rational agents should eventually realize 
that the system is locked in a perpetual cycle, so that 
more sophisticated learning rules would be needed for 
the system to converge [ij. 



8 



IV. THE CASE OF A NONZERO MUTATION 
RATE 

We now consider the effect of a nonzero mutation rate 
H > 0, maintaining our assumptions of an infinite popu- 
lation and non-discriminatory selection. The salient dif- 
ference between this case and the previous case, where 
we assumed no mutations, is that type i offspring can re- 
sult from any pairing. There are, for example, two ways 
in which a type 1 offspring can result from the pairing of 
a type 1 agent with a type 2 agent: the parents can ini- 
tially produce type 1 offspring (probability U12) which do 
not subsequently mutate (probability 1 — ^), or they can 
initially produce type 2 offspring (probability U21) which 
mutate into type 1 offspring (probability /i/2, since it is 
equally likely that they will mutate into type 3 agents). 
Considering all possible pairings by which offspring of 
each type may be produced, and again relying on the 
infinite population assumption, we obtain the following 
(simplified) update formula for the population state: 



' Pl,t+1 = 


(1- 


f) 


{pI- 


h 2pitP2tWl2 - 


1- 2pitP3tMi3) - 


^ 2 


P2,t+1 = 


(1- 


f) 


{plt- 


1- 2pitP2tU21 - 


1- 2p2tP3tM23) - 


^ 2 


. P3,t+1 = 


(1- 


f) 


{pI~ 


h 2pitP3tU31 ~ 


l~ 2p2tP3tM32) - 


^ 2 



2 ■ 

(3) 



In a biological context, we can expect mutation rates to 
be very low, i.e. fi 1 [see e.g. [Hjli^l; here, the weakest 
restriction we shall assume is fi < 2/3, though typical 
mutation rates will be much lower. Note that if /Lt > 2/3, 
wc have a perverse scenario where complete dominance 
of one type in a given period (say, pu = 1) results in 
that type being the minority in the next period (pi,t+i < 
P2,t+i,P3,t+i)- We shall see that the nonzero mutation 
rate renders finding and characterizing the fixed points 
more complicated than before - for example, the ^/2 
term at the far right of each equation in ([3]) prevents us 
from factorizing pn, if nonzero, out of the right hand side 
of the i"^ equation. 

Before a more rigorous analysis, we might make some 
initial remarks and hypotheses regarding the fixed points 
of the system, and their stability, under ([3]): 

1. The pure states p = are no longer fixed points: 
a population comprising only type i agents, will, in 
the initial reproduction stage, produce only type i 
agents, but will see a proportion fi of these offspring 
mutate to other types, rendering the resultant pop- 
ulation state 'mixed'. 

2. If U is such that would be an ES fixed point 
in the case without mutations (Thm. [2]), we might 
expect there to be an ES fixed point (or at least 
an invariant set) near under the dynamics that 
allows for mutations. 



3. If L'^ is such that there would be an interior fixed 
point in the case without mutations (Thm. [3]), then 
we might expect a similar fixed point in the case 
with mutations. It is not clear whether this point, 
if it were to exist, would be stable or not. 

We begin the more detailed analysis by rewriting ([3]), 
taking into account the restrictions pi + P2 + P3 = 1 
and Uij = 1 — Uji] under these restrictions, the third 
equation turns out to be solved identically if the first 
two are solved, and the system simplifies to: 

Pl,t+1 - Pit = Pit (1 - ^) ((1 - Plt)(2Ml3 - 1) 
f (1 - 3pit) + 2p2t{ui2 - Ui3)j 

< 

P2,t+1 - P2t = P2t (1 - ^) (2pit(l - U12 - U23) 

f (l-3p2t) + (l-p2t)(2H23-l)). 

(4) 

A. Existence of feasible fixed points 

For a fixed point, the left hand sides of the above 
equations arc zero; this leaves a system of two non- 
homogeneous quadratic forms in pi and p2 (we may drop 
the time indices): 

r [- (1 - f ) {2u^3 - 1) - 2 (1 - ^) {U,2 - U13)P2 

+ f ]pi + (l-f)(2iii3-l)p?-f =0 

< 

[- (1 - f ) (2^223 - 1) - 2 (1 - f) (1 - U,2 - U23)P1 
. +f ]p2 + (l-t)(2?i23-l)pl-^=0. 

(5) 

From this system, we expect four solutions for p = 
iPiiP2,^ — Pi ~ P2), though of course these need not lie 
on the unit simplex; this would require pi > 0,P2 > 
0,Pi +P2 < 1- 

Though analytical results have been derived, these re- 
sults are generally intractable (involving enormous ex- 
pansions which do not seem amenable to simplification 
even with the aid of a computer algebra system) and thus 
we shall present numerical results only. 

For each setup ('U12, U13, U23, /i), we ask two questions: 

1. How many, if any, fixed points lie on the unit sim- 
plex A3? 

2. For each of the fixed points that do lie on A"^, is 
the fixed point stable or unstable in terms of the 
definitions set out earlier? 

To solve for the fixed points of the system, we note 
that the above system can be written in the general form 

r ai -1- a2Pi + a3PiP2 + a^pl = , . 

\bi+ 62P2 + &3P1P2 + bip\ =0. 
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Assume for now that none of the Pi,ai, hi is zero. Then 
we may solve for p2 in the first equation in 

P2 = — — + 02 + aipi ) . (7) 

Substituting this result into the second equation in (|6]) 
and simplifying yields a quartic polynomial in pi : 

a\hi + (2aia2&4 - 0103^2)^1 + (os&i - 0203^2 - 010363 
+ 0364 + 2010464)^1 + (2020464 — 030462 — 020363)^1 
+ (0I64 - 030463)^1 = 0. (8) 

For each set of coefficients, ([8]) has four (not necessarily 
distinct, and not necessarily real) solutions, each of which 
corresponds to a solution for p2, as determined by ([7|). 
P3 is then determined by = 1 — pi — p2 • 

In this way, using a numerical approach (e.g. finding 
the eigenvalues of the polynomial's characteristic matrix) 
we may efficiently find, for each combination of param- 
eter values, the roots of We may then investigate 
whether any of these roots lies on the unit simplex. 

We looped the elements of U using a step size of 0.001, 
making sure that none of the assumptions was violated, 
and we looped /i between - but not including - its as- 
sumed limits (0, 2/3) using a step size of 0.001. The key 
result of this investigation is that, for each setup ( ex- 
actly one solution (i.e. one fixed point of the system) lies 
on A'^. Moreover, in each case, that solution lies in the 
interior of A'^. 



B. Stability 

For each setup, we have a unique fixed point on the 
unit simplex. We may analyze the stability of the fixed 
point making use of the usual Jacobian analysis, as we 
did in Sec. IIIIB] - here, though, our analysis will be nu- 
merical. For each setup, we shall numerically calculate 
the eigenvalues of the Jacobian matrix, evaluated at the 
feasible fixed point. 

1. The case of a dominant type 

We took ui2, Ui3 > i as the general case that resulted 
in ES pure states when /i = 0. With no mutations, such 
setups lead to the population state p = ei being a glob- 
ally attracting fixed point. 

In this and other cases, the limiting case — > 0+ will 
yield behavior identical to that of the case ^ ~ Q. The 
practical implication of this is that, for sufficiently low 
mutation rates, the behavior of the system will be prac- 
tically identical to that with no mutations. 

For sufficiently large mutation rates, however, and for 
certain setups, the eigenvalues of the Jacobian matrix, 
evaluated at the interior fixed point, are complex^ with 
modulus strictly less than unity, so that the behavior near 



the fixed point is no longer that of a regularly stable fixed 
point, but rather an inward spiral. For other setups, 
though, the eigenvalues are real, both with magnitude 
lower than one, so that the fixed point is regularly stable, 
as in the case with no mutations. 

In the case of a dominant type, these are the only be- 
haviors possible under the dynamics; Fig. [5] illustrates 
typical trajectories of each. 

In Fig. [21 for various mutation rates, we plot the re- 
gions in the U12-U23 plane that are associated with fixed 
points of the two stability types mentioned above. We 
fix ui3 = 0.53; thus, we are interested in setups with 
ui2 > \- Notice that the region for which the 'new' be- 
havior occurs, i.e. for which the fixed point is no longer 
regularly stable but rather a stable spiral, grows as we 
increase ji. 

2. Ttie case of no dominant type 

The general case that resulted in an unstable interior 
fixed point when /i = was taken to be U12 > 5, U13 < 
57^*23 > ^. With /i = 0, such setups lead to an unstable 
interior fixed point; the trajectories are outward spirals, 
with limit cycles at the boundary of the simplex. For 
sufficiently low mutation rates, then, this is the behavior 
of the system as well. (More precisely, for a given pay- 
off matrix U obeying the above condition, there exists 
a positive number e such that, if ^ < e, the trajectories 
around the fixed point are outward spirals whose limit 
cycle is arbitrarily close to the boundaries of the sim- 
plex.) For larger values of /i, the behavior of the system 
becomes more complex. Three types of behavior are pos- 
sible; they are described below, along with the nature of 
the eigenvalues of the Jacobian (evaluated at the fixed 
point) that diagnoses the behavior near the fixed point 
in each case. (The behavior away from the fixed point 
can not be diagnosed using the eigenvalues; for this we 
rely on numerical results.) 

1. An unstable spiral near the fixed point, spiraling 
outward to a limit cycle contained in the interior of 
A'^ . The diagnostic eigenvalue condition for the be- 
havior near the fixed point is: ImAi_2 7^ 0, |Ai,2| > 
1. 

2. A stable spiral near the fixed point. In fact, tra- 
jectories spiral inward to the fixed point from all 
points in A'^, so that it is globally attracting as 
well. Diagnostic condition: ImAi^2 7^ 0, |Ai^2| < 1- 

3. Non-spiral inward trajectories towards the fixed 
point, which we term a 'regularly stable fixed 
point'. Diagnostic condition: ImAi,2 = 0,|Ai.2| < 
1. 

Our numerical investigation reveals that, for a given 
payoff matrix U, for very low mutation rates, the be- 
havior is, as expected, that trajectories starting near the 
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(a) /i = 0.1; Ui2 = 0.60, uu = 0.55, fij;! = 0.40 (b) /i = 0.1: un = 0.80, Un = 0.60, «23 = 0.85 




FIG. 2. The two types of behavior which can arise in the case of a dominant type. Panel (a) shows typical trajectories for the 
case where the eigenvalues of the Jacobian, evaluated at the fixed point, are real, with magnitude less than unity; panel (b) 
shows typical trajectories for the case where the eigenvalues of the Jacobian, evaluated at the fixed point, are complex, with 
magnitude less than unity. For the latter trajectories, the inward spiral near the fixed point is so small as to be invisible on 
the plot. 



{a)/(=0 (b);j = 0.001 (c);t = 0.01 (d)/i = 0.1 



J 




0.6 0.7 0.8 0.9 0.6 0.7 0.8 0.9 0.6 0.7 0.8 0.9 0.6 0.7 0.8 0.9 

"12 Un ui2 uu 



FIG. 3. The case of a dominant type. Regions in the ui2-U2-i plane which result in fixed points of the various types, for various 
mutation rates, fixing uia — 0.53. The light regions correspond to regularly stable fixed points; the dark regions correspond to 
stable spirals. 



fixed point spiral outward to a limit cycle that is con- 
tained in the interior of A'^, while trajectories starting 
outside the limit cycle spiral inward towards it. As we 
increase ^ slightly, the limit cycle shrinks, though the 
behavior of the system does not change qualitatively. As 
we continue to increase fi, at some point the limit cycle 
shrinks to the fixed point. As we increase ^ past this 
value, the fixed point becomes the limit of inward spi- 
raling trajectories. (Increasing n still further - for most 
payoff matrices, to very large values - the fixed point 
eventually becomes regularly stable. Because we typi- 
cally consider low mutation rates, and because the sta- 
bility of the fixed point does not change, this result is not 
of significant interest.) 

This is diagnosed by the observation that the mag- 
nitude of complex eigenvalues of the Jacobian matrix, 
evaluated at the fixed point, decreases through unity as 



we increase fi through its critical value. 

Thus, the system undergoes a supercritical Hopf bi- 
furcation [l^: as we increase n through some critical 
value /J,*, the fixed point changes from an unstable spiral 
to a stable spiral. Moreover, for most payoff matrices, 
this occurs for relatively low values of fi. For example, 
the critical value of fi in the case ui2 = 0.7, U13 = 0.4, 
W23 = 0.6 is /i* = 0.0057. We shall see that, fixing U13, 
for small values of U12 and W23 (i.e. close to i), the criti- 
cal value of ^ is very small, while for large values of M12 
and U23 (i.e. close to 1), the critical value of ^ is larger. 

The significance of this result is clear: even very low 
mutation rates can stabilize a system that is, in the ab- 
sence of mutations, unstable. The population state in 
the case of no mutations eventually oscillates wildly be- 
tween near-pure states, but if we introduce even a very 
low mutation rate, the population state instead eventu- 
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FIG. 4. The stabilizing efTect, through a Hopf bifurcation, of mutations in the case of no dominant type. Evolution of the 
components of the population state over time for /i = and fi = 0.008 > fi* , for the case ui2 = 0.7, U13 = 0.4, U23 = 0.6. 



ally settles to a stable point of fixed proportions. Fig. |3] 
demonstrates these different behaviors, plotting the time 
evolution of the population proportions for the case of 
a zero mutation rate and the case of a mutation rate 
slightly above the critical value. Fig. [5] illustrates the 
Hopf bifurcation in the phase plane for the same payoff 
matrix. 

Fixing = 0.4, and for various mutation rates, we 
plot in Fig. [B] the regions in the U12-U23 plane for which 
the types of behavior described above arise. The region 
for which the original (zero mutation rate) behavior, that 
of an unstable spiral, occurs, shrinks as we increase /i. 
The boundary of the region within each plot corresponds 
to the payoff matrices for which the value of /i in ques- 
tion is the critical value. Notice that the payoff matrices 
with the largest critical mutation rates are those with the 
largest values for U12 and U23. 



V. CONCLUSIONS 
A. Review 

We have presented a study of a purely competitive dy- 
namics for 3-type games, arguing that such a dynam- 
ics represents a characterization of how certain systems, 
both natural and artificial, are governed. 

Throughout, we maintained the assumptions of an infi- 
nite population, and that the probability of advancement 



of a certain type in an engagement with another is pro- 
portional to that type's payoff in the engagement. We 
further assumed that, in each period, each agent is se- 
lected for pairing, though the general model developed 
in Sec. |lT] allows for more general selection mechanisms, 
and indeed for all these assumption to be relaxed. 

Initially ignoring the possibility of mutations, we 
showed that two types of behavior are possible for the 
population state's evolution, corresponding to two dif- 
ferent types of payoff matrix. The first, where one of 
the types is dominant (in the sense that in an engage- 
ment with another type, it always receives the higher 
payoff) the population state eventually comprises only 
agents of the dominant type (unless it initially comprised 
only agents of another type). If no type is dominant, on 
the other hand, the components of the population state 
(assuming it is not initially at the mixed- type fixed point, 
the existence of which is guaranteed for payoff matrices 
of this type) eventually fluctuate, in procession, between 
being very close to zero and very close to unity. 

Allowing for a nonzero mutation rate, our analysis be- 
came predominantly numerical. We showed that, in the 
case of a dominant type, the behavior of the system does 
not differ significantly from that under the assumption of 
a zero mutation rate. In particular, the population state 
still tends to a globally attracting fixed point near domi- 
nant type's pure state. For the case of no dominant type, 
the dynamics of the system does, in some cases, alter sig- 
nificantly. For very low mutation rates, as expected, the 
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(a) = 0.002 {b)/i = 0.007 




FIG. 5. The stabilizing effect, through a Hopf bifurcation, of mutations in the case of no dominant type. Trajectories for (a) 
At = 0.002 < fj,* and (b) = 0.007 > in both cases for ui2 = 0.70, ui3 = 0.40, U23 = 0.60. 



(a)/j = (b)/i = 0.001 (c)/i = 0.01 (d)/( = 0.1 
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FIG. 6. The case of no dominant type. Regions in the U12-U23 plane that result in fixed points of the various types, for various 
mutation rates, fixing U13 = 0.4. The lightest regions correspond to unstable spirals; the mid-tone regions correspond to stable 
spirals; and the darkest regions correspond to regularly stable fixed points. 



system behaves as it docs under the assumption of no 
mutations; that is, the population state, if starting near 
the fixed point in the interior, spirals outwards tovifards 
a Umiting cycle, so that, again, the proportions of the 
types eventually fluctuate, in procession, w^ith significant 
amplitude. However, as we increase the mutation rate 
for a given setup, at some point, the interior fixed point, 
previously an outward spiral, becomes an inward spiral: 
the system undergoes a Hopf bifurcation at some critical 
value of The result is that, for mutation rates larger 
than the critical value, the fixed point is stable; the popu- 
lation proportions eventually settle down to fixed values, 
rather than continuing to fluctuate, as in the case of zero 
or very low mutation rates. Moreover, this critical value 
was shown to be relatively low for most setups, so that 
this effect should not be thought of as pathological. That 
even a very low mutation rate can alter the behavior of 
the system so signiflcantly - indeed, entirely reversing 
its instability - is a result of both theoretical and prac- 
tical interest, and it would certainly be worth exploring 



the significance of this result in the context of real- world 
dynamical systems. 



B. Possibilities for future study 

In considering the possibilities for extension of the 
analysis carried out here, we naturally turn to the sim- 
plifying assumptions we have made, and ask whether the 
relaxation of any of them could yield further insight into 
these dynamics. 

First, we assumed throughout that 9 ~ I. Increasing 
the value of this parameter serves to favor the stronger 
type in a pairing, in the sense that the probability that 
the stronger type propagates its type to its offspring in- 
creases as we increase 6. Of interest is the limiting case 
6 = 00, where in a given pairing, the offspring receive 
with certainty the type of the stronger parent. Intuition 
suggests that, in the case of a dominant type, that type's 
predominance in terms of its proportion in the long term 
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population state would be increased relative to the case 
where 9 = 1 (for nonzero mutation rates, of course ~ for 
a zero mutation rate, we would expect the long term out- 
come to be that type's pure state, regardless of 9). In the 
case of no dominant type, it is not clear whether the dy- 
namics observed for 9 = 1 would be preserved for 9 = oo, 
though a numerical investigation would be revealing in 
this regard. 

Second, our assumption that in each period, each agent 
is selected for mating {(pi = 1), can be weakened to one 
where an agent's selection for mating is dependent on 
its fitness. In this sense, the dynamics would no longer 
be purely competitive, since an agent's fitness, or average 
payoff, depends in part on its payofi^ in engagements with 
agents of its own type. This would align the dynamics 
more with the replicator dynamics [H ; whether the resul- 
tant behavior would more closely resemble that for the 
dynamics we considered here or the replicator dynamics 
could be investigated numerically. 

Finally, the assumption of an infinite population was 
maintained throughout, allowing probabilities and real- 
ized proportions to be equated. Of interest would be an 



investigation of the effect of a finite population on the 
results above; we might expect the behavior of very large 
populations to resemble that of an infinite population, 
but would small populations also exhibit such behavior? 
Indeed, a preliminary analysis carried out by the authors, 
making use of simulations of finite population cases, sug- 
gests that for some setups, the long run behavior of the 
population state is qualitatively different for small popu- 
lations (say, N = 20) than larger ones (say, N = 200). A 
more detailed investigation of the finite population case 
will be carried out in later work. 



ACKNOWLEDGMENTS 

The authors thank Martin Wittenberg and Drew Fu- 
denbcrg for helpful comments, and acknowledge with 
gratitude the University of Cape Town and the National 
Research Foundation for the provision of financial sup- 
port. 



D. Fudenberg and D. Levine, The Theory of Learning in 
Games (The MIT Press, Cambridge, MA, 1998). 
L. Samuelson, Evolutionary Games and Equilibrium Se- 
lection (The MIT Press, Cambridge, MA, 1998). 
P. Taylor and L. Jonker, Math. Biosci., 40, 145 (1978). 
J. Hofbauer and K. Sigmund, Evolutionary Games 
and Population Dynamics (Cambridge University Press, 
Cambridge, UK, 1998). 

J. Hofbauer and K. Sigmund, Bull. Am. Math. Soc, 40, 
479 (2003). 

D. L. Hartl and A. Clark, Principles of population qenet- 



ics (Sinaucr Associates, Inc., Sunderland, MA, 1997). 



(iUniverse, Inc., Lincoln, NE, 2004). 
[16] N. E. Miller and J. DoUard, 

\Social Learning and Imitation , edited by B. G. Galef 

and C. M. Hayes, Vol. 39 (Yale University Press, 1941) 

pp. 128-129.^ 

[17] K. H. Schlag, [TFfey Imitate, and if so, How? Exploring a Model of So 

Discussion Paper Serie B (University of Bonn, Germany, 

1994). 

[18] K. Laland, [Learning fc Behavior, 32, 4 (2004) | ISSN 

1543-4494. 

[19] C. Castellano, S. Fort unato, and V. Loreto, 



M. Kandori, G. J. Mailath, and R. Rob, 

Econometrica, 61, 29 (1993) 

G. W. Rowc, J. Thcor. Biol., 117, 291 (1985). 

I. M. Bomze and R. Burger, Games Econ. Behav., 11, 

146 (1995). 

D. Fudenberg and L. A. Imhof, 
|J. Econ . Theor y, 131, 251 (2006)[ 
S. Wright, Proc. Natl. Acad. Sci., 23, 307 (1937). 
C. Darwin, The Descent of Man, and Selection in Rela- 
tion to Sex (John Murray, London, 1871). 
R. Thornhill and J. Alcock, The evolution of insect mat- 
ing systems (Harvard University Press, Cambridge, MA, 
1983). 

M. Andersson, Sexual selection (Princeton University 
Press, Princeton, NJ, 1994). 

T. Donelson and F. Lotierzo, More Tales From Ringside 



Rev. Mod 



Phys., 81, 591 (2009) . 
[20] J. Maynard Smith, Evolution and the Theory of Games 

(Cambridge University Press, Cambridge, UK, 1982). 
[21] I. Eshel, J. Math. Biol ., 34, 485 (1996) , ISSN 0303-6812, 

10.1007/BF02409747. 
[22] V. L. Kocic and G. Ladas, Global Behavior of Nonlin- 
ear Difference Equations of Higher Order with Applica- 
tions (Kluwer Academic Publishers, AA Dordrecht, The 
Netherlands, 1993). 
[23] M. W. Nachman and S. L. Crowell, Genetics, 156, 297 

(2000), ISSN 1943-2631. 
[24] R. L. Haupt and S. E. Haupt, Practical Genetic Algo- 
rithms (Wiley-Interscience, Hoboken, NJ, 2004) ISBN 
0471455652. 

[25] S. Strogatz, Nonlinear Dynamics and Chaos (Westview 
Press, Boulder, CO, 1994). 



